All Questions
Tagged with model-evaluationspredictive-modeling
21 questions
0votes
0answers
17views
Impact of Adding Imbalanced Data on Model Performance for Different Groups
Suppose I initially have a dataset with 50 samples of type A and 50 samples of type B, each with several features. I built a neural network model using this data and recorded the prediction accuracy ...
1vote
1answer
123views
How do I work with time-series data of temperature?
So I have some equipment temperature and i have outside temperature (both are collected daily) and I want to predict the equipment temperature. However, I'm new to this and unsure about which model to ...
- 11
0votes
1answer
664views
How to compare test vs train model performance
When comparing the test vs train model performance to ensure no overfitting (e.g., using AUC ROC as an example), is it better to select the model with the largest test score, or the model with the ...
0votes
0answers
21views
What are some approaches to improve the classification of ONE particular instance of interest?
I'd like to know if there are some methods to correct a specific misclassified instance of interest (e.g. in a Logistic Regression or Random Forest). Like maybe increasing the error for that ...
- 111
0votes
1answer
14views
Metrics in pediction different than evaluation
In general when you have already evaluated your model on unseen data (test set) and its RMSE is different than predictions RMSE, is it ok ? How much difference is fine and how to know that ?
0votes
0answers
52views
How to evaluate model accuracy at tail of empirical distribution?
I am making a nonlinear regression on stationary dependent variable and I want to precisely forecast extreme values of this variable. So when my model predicts extreme values I want them to be highly ...
1vote
1answer
5kviews
How could we interpret a SI Scatter Index and RMSE?
SI is RMSE divided by the average value of the observed values (or the predicted values? am confused)? is SI = 25% acceptable? (is the model good enough? )
- 11
1vote
1answer
502views
In general, what are precision, recall, F1 that are reported in papers?
I used classification_report in sklearn library And, the picture below shows evaluation on my model (anomaly detector) In general, what are precision, recall, F1 ...
0votes
1answer
26views
How to interprete the feature significance and the evaluation metrics in classification predictive model?
Consider a experiment to predict the Google-Play apps rating using a Random-Forest classifier with scikit-learn in Python. Three attributes 'Free', 'Size' and 'Category' are utilized to predict the ...
- 177
3votes
3answers
4kviews
Best common metric for comparing classic time series forecasting methods (ARIMA/Prophet) with ML approach?
I am new to time series forecasting and looking to compare the performance of ARIMA/Prophet with an XGBoost model in predicting future stock market values based on historical stock market data and ...
- 197
1vote
1answer
3kviews
How to Maximize recall for Minority class?
I have a dataset with 4.7k records and 60 features. 1558 records of indication label 1 and 3554 records indicating label 0. Am ...
- 2,715
2votes
1answer
53views
How can I get an algorithm to have an evalutation metric based on aggregate predictions?
Let's say I have a model that makes a prediction per individual. An example data set is below. Normally, evaluation metrics (for example within the XGBoost algorthim), are used at the individual ...
1vote
0answers
177views
Error-analysis and evaluation of a model using Python?
My method of evaluating a model is the following : Split the training data set and do cross validation to obtain an accuracy of my model on my cross validation data set. Use the parameters that gave ...
- 2,124
1vote
1answer
409views
Logistic Regression - ROC curve plots Sensitivity vs Specificity instead of (1-Specificity)
I am new to Machine Learning and have been doing some practice on Logistic Regression. To evaluate the models, I've been trying to create some ROC plots. The package that i used is pROC. The model ...
2votes
1answer
99views
Ranking ATM based on Utilization and Economic Data (Scoring/Rank Model)
I have a sample data of around 10 ATM's Locations along with their Utilization Count (Deposits, Withdrawals and Others) for the past 3 months. I am planning to collect additional data such as nearby ...
- 65
